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determining the positions of the recognised words in said processing application program means, 
means for monitoring changes in the positions of the recognised words, and [link] means for 
forming link data linking the audio [identifiers] data to the [character component positions in the 
character string and for updating said link data after processing to maintain the link between the 
audio identifiers and the character component positions in the processed character string] 
recognised words, said link data comprising the audio identifiers and the determined positions of 
corresponding recognised words, said interface application program means including means for 
updating said link data in response to monitored changes in positions of the recognised words : 

display means for displaying the [characters] recognised words received and processed by 
said processing application program means; 

user operable selection means for selecting [characters] at least one word in the displayed 
[characters for audio playback, where said link data identifies] words, said interface application 
program means including means for identifying any [selected] audio components, if present, 
which are linked to the at least one selected [characters] word : and 

audio playback means for playing back [the] any [selected] identified audio components 
in the order of the [character component] word positions in the [character] word string or the 
processed [character] word string. 

2. [Once Amended] Data processing apparatus as claimed in claim 1 wherein said 
storage means also stores the [characters,] recognised words and the link data [and the audio 
data], [and] the apparatus including storage reading means for reading the stored [characters] 
recognised words into said processing application program means and for reading the stored link 
data for use by said [processing] interface application program means [and said link means, 
whereby said user operable selection means can select displayed characters for audio playback 
and said audio playback means reads and plays back the audio components corresponding to the 
selected characters]. 

3. [Once Amended] Data processing apparatus as claimed in claim 1 including user 
operable correction means for selecting [and correcting any] a displayed recognised [characters] 
word which [have] has been incorrectly recognised[J; correction audio playback means for 
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controlling said audio playback means to play back any audio component corresponding to the 
selected [characters] word to aid correction; and speech recognition update means for sending the 
corrected [characters] word and the audio identifier for the audio component corresponding to the 
corrected [character] word to the speech recognition engine. 

4. [Once Amended] Data processing apparatus as claimed in claim 3 wherein said 
recognition data includes alternative [characters] words , said display means including means to 
display a choice list comprising the alternative [characters] words , and said selecting and 
correcting means [including] includes means to select one of the alternative [characters] words or 
to enter a new [character] word . 

5. [Once Amended] Data processing apparatus as claimed in claim 1 wherein said [link 
means comprises memory means storing a list of character locations in the character string and] 
audio identifiers comprise a list of positions of the corresponding audio components in the audio 
data. 

6. [Once Amended] Data processing apparatus as claimed in claim 5 wherein said 
[character] word string is formed of a plurality of separately dictated passages of [characters] 
words , [the apparatus including audio] said storage means [storing] stores said audio data for 
each dictated passage of [characters] words in a separate file, and said memory means [storing] 
stores a list identifying the files and positions in the files of the audio components in said audio 
data corresponding to the word locations in the [character] word string. 

7. [Once Amended] Data processing apparatus as claimed in claim 1 wherein said 
recognition data includes recognition status indicators to indicate whether each recognised 
[character] word is a [character] word finally selected as recognised by said speech recognition 
engine or a [character] word which is the most likely at that time but which is still being 
recognised by said speech recognition engine, the apparatus including status detection means for 
detecting said recognition status indicators, and display control means to control said display 
means to display [characters] words which are still being recognised differently to [characters] 
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words which have been recognised, said [link] interface application program means being 
responsive to said recognition status indicators to link the recognised [characters] words to the 
corresponding audio component in the audio data. 

8. [Once Amended] Data processing apparatus as claimed in claim 1 including 
contextual update means operable by a user to select recognised [characters] words which are to 
be used to provide contextual correcting parameters to said speech recognition engine, and to 
send said contextual correcting parameters to said speech recognition engine. 

9. [Once Amended] Data processing apparatus as claimed in claim 1 wherein said 
recognition data includes a likelihood indicator for each [character] word in the [character] word 
string indicating the likelihood that the [character] word is correct, and said link means stores the 
likelihood [confidence] indicators, the apparatus including 

automatic error detection means for detecting possible errors in recognition of 
[characters] words in the recognised [characters] words by scanning the likelihood indicators in 
said link means for the recognised [characters] words and detecting if the likelihood indicator for 
a [character] word is below a likelihood threshold, whereby said display means highlights the 
[character] word having a likelihood indicator below the likelihood threshold; 

second user operable selection means for selecting a [character] word to replace an 
incorrectly recognised [character] word highlighted in the recognised [characters] words : and 

correction means for replacing the incorrectly recognised [character] word with the 
selected [character] word to correct the recognised [characters] words . 

10. [Once Amended] Data processing apparatus as claimed in claim 1 including 
file storage means for storing the recognised [characters] words in a file; 

means for selectively disabling one of the receipt of the recognised [characters] words by 
said processing application program means and the recognition of speech by said speech 
recognition engine for a period of time, means for storing the audio data for the period of time in 
said storage means as an audio message associated with the file; and 

storage reading means for reading said file for input to said processing application 
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program means, and for reading said audio message for playback by said audio playback means. 

1 1 . [Once Amended] Data processing apparatus as claimed in claim 10 wherein said 
storage reading means is controllable by a user to read said audio message at any time after said 
file has been input to said processing application program means until said processing 
application program means is no longer processing said file. 

12. [Once Amended] Data processing apparatus as claimed in claim 1 wherein said user 
operable selection means is operative to allow a user to select to playback the audio data for the 
most recent passage of dictated [characters] words , or to select [characters] words and play back 
the corresponding audio components. 



a data processing apparatus, the data processing apparatus comprising: 
input means for receiving recognition data [and corresponding audio data] from a speech 
recognition engine and corresponding audio data , said recognition data including a string of 
recognised [characters] words and audio identifiers identifying audio components corresponding 
to [a character component of the] each recognised [characters] word : 

[processing] interface application program means comprising means for receiving [and 
processing] the input recognised words, means for placing the recognised words into positions in 
text in a processing application program means to allow the processing of the recognised words 
to change the positions of [characters to at least one of replace, insert move and position] the 
recognised [characters] words to form a processed [character] word string[:] . means for 
determining the positions of the recognised words in said processing application program means, 
means for monitoring changes in the positions of the recognised words, and [link] means for 
forming link data linking the audio [identifiers] data to the [character component positions in the 
character string and for updating said link data after processing to maintain the link between the 
audio identifiers and the character component positions in the processed character string] 
recognised words, said link data comprising the audio identifiers and the determined positions of 
corresponding recognised words, said interface application program means including means for 




[Once Amended] 



A data processing arrangement comprising: 
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updating said link data in response to monitored changes in positions of the recognised words : 

storage means for storing said recognition data and audio data received from said input 
means, and for storing said link data; 

display means for displaying the [characters] recognised words received and processed by 
said processing application program means; 

user operable selection means for selecting [characters] at least one word in the displayed 
[characters for audio playback, where said link data identifies] words, said interface application 
program means including means for identifying any [selected] audio components, if present, 
which are linked to the at least one selected [characters] word : and 

audio playback means for playing back [the] any [selected] identified audio components 
in the order of the [character component] word positions in the [character] word string or the 
processed [character] word string; and 

an editor work station comprising: 

data reading means for reading the [characters] words , link data, and audio data from said 
data processing apparatus; 

editor processing means for processing the [characters] words : 

editor link means for linking the audio data to the [character component position] word 
positions using the link data; 

editor display means for displaying the [characters] words being processed; 

editor correction means for selecting and correcting any displayed [characters] words 
which have been incorrectly recognised; 

editor audio playback means for playing back [any] an audio component corresponding to 
[the] any selected [characters] words to aid correction; 

editor speech recognition update means for storing the corrected [characters] words and 
the audio identifier for the audio component corresponding to the corrected [character] word in a 
[character] word correction file; and 

data transfer means for transferring the [character] word correction file to said data 
processing apparatus for later updating of models used by said speech recognition engine; 

said data processing apparatus including correction file reading means for reading said 
[character] word correction file to pass the data contained therein to said speech recognition 

SY 
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[Once Amended] 



A data processing arrangement as 



claimed in claii 



wherein said 



recognition data includes alternative [characters] words , said editor display means including 
means to display a choice list comprising the alternative [characters] words , and said editor 
correcting means [including] includes means to select one of the alternative [characters] words or 
to enter a new [character] word . 



[Once Amended] A data processing arrangement as claimed in claim Y$ including editor 
contextual update means operable by a user to select recognised [characters] words which are to 
be used to provide contextual correcting parameters to said speech recognition engine of said 
data processing apparatus, and to store said contextual correcting parameters in a contextual 



said data transfer means being responsive to the contextual correction file to transfer the 
contextual correction file to said data processing apparatus for later updating of models used by 
said speech recognition engine; 

said correction file reading means of said data processing apparatus being responsive to 
the contextual correction file to read the contextual correction file to pass the data contained 
therein to said speech recognition engine. 



[Once Amended] A data processing arrangement as claimed in claim-B wherein said 
recognition data includes a likelihood indicator for each [character] word in the [character] word 
string indicating the likelihood that the [character] word is correct, and said link data includes the 
likelihood indicators, said editor work station including editor automatic error detection means 
for detecting possible errors in recognition of [characters] words in the recognised [characters] 
words by scanning the likelihood indicators in said recognition data for the [characters] words 
and detecting if the likelihood indicator for a [character] word is below a likelihood threshold, 
whereby said editor display means highlights [characters] words having a likelihood indicator 
below the likelihood threshold; 

editor selection means for selecting a [character] word to replace an incorrectly 




correction file; 
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recognised [character] word highlighted in the text; and 

second editor correction means for replacing the incorrectly recognised [character] word 
with the selected [character] word to correct the recognised [characters] words . 



Jfi. [Once Amended] A data processing arrangement as claimed in claim-« wherein said 
data processing apparatus includes file storage means for storing the recognised [characters] 
words in a file; 

means for selectively disabling one of the receipt of the recognised [characters] words by 
said processing application program means and the recognition of speech by said speech 
recognition engine for a period of time; [with] 

means for storing the audio data [for] during the period of time in said storage means as 
an audio message associated with the [document] file; and 

storage reading means for reading said [document] file for input to said processing 
application program means, and for reading said audio message for playback by said audio 
playback means; 

said editor work station including audio message reading means for reading the audio 
message associated with [characters] words being processed by said editor processing means for 
playback by said editor audio playback means. 



[Once Amended] A data processing arrangement as claimed in claim-*? wherein said 
audio message reading means is controllable by a user to read said audio message at any time the 
associated [characters] words are being processed by said editor processing means. 



19. [Once Amended] An editor work station for use with the data processing 
arrangement as claimed in claim-^*, said editor work station comprising: 

data reading means for reading the [characters] words , link data, and audio data from said 
data processing apparatus; 

editor processing means for processing [characters] words : 

editor link means for linking the audio data to the [character component position] word 
positions using the link data; 
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editor display means for displaying the read [characters] words : 

editor correction means for selecting and correcting any displayed [characters] words 
which have been incorrectly recognised; 

editor audio playback means for playing back any audio component corresponding to the 
selected [characters] words to aid correction; 

editor speech recognition update means for storing the corrected [character] word and the 
audio identifier for the audio component corresponding to the corrected [character] word in a 
character correction file; and 

data transfer means for transferring the [character] word correction file to said data 
processing apparatus for later updating of models used by said speech recognition engine. 



^0. [Once Amended] An editor work station as claimed in claim i9 wherein said 
recognition data includes alternative [characters] words , said editor display means including 
means to display a choice list comprising the alternative [characters] words , and said editor 
correcting means [including] includes means to select one of the alternative [characters] words or 
to enter a new [character] word . 



JH. [Once Amended] An editor work station as claimed in clairn^ including editor 
contextual update means operable by a user to select recognised [characters] words which are to 
be used to provide contextual correcting parameters to said speech recognition engine of said 
data processing apparatus, and to store said contextual correcting parameters in a contextual 
correction file; 

said data transfer means being responsive to the contextual correction file to transfer the 
contextual correction file to said data processing apparatus for later updating of models used by 
said speech recognition engine; 

said correction file reading means of said data processing apparatus being responsive to 
the contextual correction file to read the contextual correction file to pass the data contained 
therein to said speech recognition engine. 

22: [Once Amended] An editor work station as claimed in clainH^ wherein said 
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recognition data includes a likelihood indicator for each [character] word in the [character] word 
string indicating the likelihood that the [character] word is correct, and said link data includes the 
likelihood indicators, said editor work station including editor automatic error detection means 
for detecting possible errors in recognition of [characters] words in the recognised [characters] 
words by scanning the likelihood indicators in said recognition data for the [characters] words 
and detecting if the likelihood indicator for a [character] word is below a likelihood threshold, 
whereby said editor display means highlights characters having a likelihood indicator below the 
likelihood threshold; 

editor selection means for selecting a [character] word to replace an incorrectly 
recognised word highlighted in the [character] word string; and 

second editor correction means for replacing the incorrectly recognised [character] word 
with the selected [character] word to correct the recognised [text] word. 



23 . [Once Amended] A data processing method comprising [the steps of] : 

receiving recognition data [and corresponding audio data] from a speech recognition 
engine and corresponding audio data in an interface application program , said recognition data 
including a string of recognised [characters] words and audio identifiers identifying audio 
components corresponding to [text components in the recognised text] each recognised word ; 
storing the audio data; 

inputting the recognised [characters to] words into a processing application program 
which places the words in positions in the application, and which processes th e recognised words 
such that positions of the recognised words are changed [processor for the processing of the 
characters to at least one of replace, insert move and position the characters] to form a processed 
[character] word string; 

using the interface application program to determine the po sitions of the recognised 
words in the processing application program, monitor changes in th e positions of the recognised 
words, and to form [forming] link data linking the audio [identifiers] data to the [character 
component positions in the characters and updating said link data after processing to maintain the 
link between the audio identifiers and the character component positions in the processed 
character string] recognised words, said link data comprising th e audio identifiers and the 
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determined positions of corresponding recognised words, said link data being updated in 
response to monitored changes in positions of the rec ognised words: 

displaying the [characters] recognised words input to and processed bv the processor 
application : 

selecting at least one displayed [characters for audio playback] word , whereby said link 
data identifies any [selected] audio components, if present, which are linked to the at least one 
selected [characters] word : and 

playing back [the] any selected audio components in the order of the [character 
component] word positions in the [character] word string. 

[Once Amended] A method as claimed in claim-23 wherein the [characters,] words 
and the link data [and the audio data is] are also stored, the method including [the step of] : 
reading the stored [characters] words into the processor application program and reading the 
stored link data[, whereby any of the read characters can be selected for audio playback, the read 
back data links the selected read characters to any corresponding stored audio data, and 
corresponding audio data is read and played back]. 

[Once Amended] A method as claimed in claim-23 including [the steps of] : selecting 
any displayed [characters] words which [has] have been incorrectly recognised, playing back 
[any] an audio component corresponding to [the] any selected [characters] words to aid 
correction, correcting the incorrectly recognised [characters] words , and sending the corrected 
[characters] word and audio identifier for the audio component corresponding to the corrected 
[character] word to the speech recognition engine. 

# op 

26. [Once Amended] A method as claimed in clairo25 wherein said recognition data 
includes alternative [characters,] words: the method [including the step of] includes displaying a 
choice list when any displayed [characters] words have been selected for correction, said choice 
list comprising said alternative [characters] words : and said [correcting step] correction of the 
incorrectly recognised words comprises selecting one of the alternative [characters] words or 
inputting a new [character] word . 
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J¥l. [Once Amended] A method as claimed in claim 33 wherein said [link data comprises 
a list of character locations in the characters and] audio identifiers comprise a list of positions of 
the corresponding audio components in the audio data. 

28. [Once Amended] A method as claimed in clainx^Twherein said [text] word string is 
formed of a plurality of separately dictated passages of [characters] words , the method including 
[the steps of] : storing said audio data for each dictated passage of [characters] words in separate 
files, said link data including a list identifying the files and positions in the files of the audio 
components in said audio data corresponding to the word locations in the [characters] word 
string . 

op 

2$. [Once Amended] A method as claimed in claim-2^ wherein said recognition data 
includes recognition status indicators to indicate whether each recognised [character] word is a 
[character] word finally selected as recognised by said speech recognition engine or a [character] 
word which is the most likely at that time but which is still being recognised by said speech 
recognition engine, the method including [the steps of] : detecting said recognition status 
indicators, displaying [characters] words which are still being recognised differently to the 
[characters] words which have been recognised, and forming said link data by linking the 
positions of the recognised [characters] words in the [characters] word string to the positions of 
the corresponding audio components in the audio data. 

3Q. [Once Amended] A method as claimed in claint^ including [the steps of] : 
selecting recognised [characters] words which are to be used to provide contextual correcting 
parameters to said speech recognition engine, and sending the contextual correcting parameters 
to said speech recognition engine. 

<%\ . [Once Amended] A method as claimed in claim 33 wherein said recognition data 
includes a likelihood indicator for each [character] word in the [characters] word string indicating 
the likelihood that the [character] word is correct, the method including [the steps of] : 

detecting possible errors in recognition of [characters] words in the [characters] word 
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string by scanning the likelihood indicators for the [characters] recognised words , and detecting 
if the likelihood indicator for a [character] word i^6elo^ likelihood threshold; 

highlighting the [character] word having a likelihood indicator below the likelihood 
threshold; 

if the highlighted [character] word is an incorrectly recognised [character] word , selecting 
a [character] word to replace an incorrectly recognised [character] word highlighted in the 
[characters] recognised words : and 

replacing the incorrectly recognised [character] word with the selected [character] word to 
correct the [characters] recognised words . 

,32. [Once Amended] A method as claimed in claims including [the steps of ] : 
storing the [characters] recognised words as a file; 
selectively disabling one of the importation of recognised [characters] words into the 
processor application program and the recognition of speech by said speech recognition engine 
for a period of time; 

storing the audio data [for] during the period of time as an audio message associated with 

the file; 

at a later time, reading said file for input to the processor application program : and 
allowing a user to select whether to read and playback said audio message associated with 
said file. 




7j\ 

[Once Amended] A method as claimed in claim*23 including [the step of] allowing a 
user to select to playback the audio data for the most recent passage of dictated [characters] 
words . 

-3-5 . [Once Amended] A method of processing data comprising [the steps^of] : 

at an author work station, carrying out the method as claimed in claim-23- wherein the 

[characters] recognised words , the link data and the audio data [is] axe stored; and 

at an editor work station, obtaining the stored [characters] recognised words, link data 

and audio data from the author work station; 



AMENDMENT AND RESPONSE 



Page 14 



Patent Application Serial No. 08/720,373 

inputting the [characters] recognised words into a processor application program : 
linking the audio data to the [character component] word positions using the link data; 
displaying the [characters] words being processed; 

selecting any displayed [characters] words which have been incorrectly recognised; 
playing back any audio component corresponding to the selected [characters] words to aid 
correction; 

correcting the incorrectly recognised [characters] words : 

storing the corrected [characters] word and the audio identifier for the audio component 
corresponding to the corrected [character] word in a [character] word correction file; and 

transferring the [character] word correction file to the author work station for later 
updating of models used by said speech recognition engine; 

wherein, at a later time, said [character] word correction file is read at said author work 
station to pass the data contained therein to said speech recognition engine for updating of said 



Jd6. [Once Amended] A method as claimed in claim-3-5 wherein said recognition data 
includes alternative [characters] words, and the [correcting step] correction o f the incorrectly 
recognised words at said editor work station, [comprising the steps of] comprises: displaying a 
choice list comprising the alternative [characters] words , and selecting one of the alternative 
[characters] words or entering a new [character] word . 



3?. [Once Amended] A method as claimed in claim %5 including [the steps] at said 
editor work station [of]: selecting recognised [characters] words which are to be used to provide 
contextual correcting parameters to said speech recognition engine at said author work station; 
storing said contextual correcting parameters in a contextual correction file; and 
transferring said contextual correction file to said author work station for later updating of 
models used by said speech recognition engine; and 

at said author work station, at a later time, reading the transferred contextual correction 
file and passing the data contained therein to said speech recognition engine. 



models. 



4\ 
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[Once Amended] A method as claimed in claim -35 wherein said recognition data 
includes a likelihood indicator for each [character] word in the [characters] word string indicating 
the likelihood that the [character] word is correct, the method including [the steps] at said editor 
work station [of]: 

automatically detecting possible errors in recognition of [characters] words by scanning 
the likelihood indicators for the [characters] words : 

detecting if the likelihood indicator for a [character] word is below a likelihood threshold, 
whereby [characters] words having a likelihood indicator below the likelihood threshold are 
displayed highlighted; 

selecting a [character] word to replace an incorrectly recognised [character] word 
highlighted in the [characters] word string : and 

replacing the incorrectly recognised [character] word with the selected [character] word to 
correct the [characters] recognised words . 




JO 

"3D. [Once Amended] A method as claimed in claim^5 wherein the method includes [the 
steps of]: 

at said author work station, storing the [characters] words as a file; 

selectively disabling one of the importation of recognised [characters] words into the 
processor application program and the recognition of speech by said speech recognition engine 
for a period of time; 

storing the audio data for the period of time as an audio message associated with the file; 
at a later time, reading said file for input to the processor application program : and, 
at said editor work station, reading the audio message associated with the file being 
processed by the processor application program , and playing back the read audio message. 

[Once Amended] A method as claimed in claims wherein the audio message can 
be read and played back at any time said file is open in the processor application program . 

M . [Once Amended] A method as claimed in claim-35 including [the step of] allowing a 

user of the editor work station to playback the audio data for the most recent passage of dictated 

O 
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[characters] words . 



jfe. [Once Amended] A data processing arrangement as claimed in claim -13 comprising 
a plurality of said data processing apparatus connected to a network, and at least one editor work 
station, wherein each editor work station can access and edit stored [characters] words and audio 
data on a plurality of said data processing apparatus. 

7 " ~ 

,52. [Once Amended] A computer usable medium having computer readable instructions 
stored therein for causing a processor in a data processing apparatus to process recognition 
signals defining a string of [characters] recognised words and corresponding audio data signals to 
display the [characters] words and selectively play the audio data, the instructions comprising 
instructions for: 

a) causing the processor to receive the recognition signals from a speech recognition 
engine and the audio data signals , the recognition signals including a string of recognised 
[characters] words and audio [identifier] identifiers identifying [the] audio components 
corresponding to each [character components in the] recognised [characters] word; 

b) causing the processor to store the audio data: 

[b) ] c) causing the processor to [process the signals to manipulate the characters] 
implement an interface application program which receives the recognised words and places the 
words in positions in a processing application program which can process the recognised words 
such that the positions of the recognised words are changed to form a processed word string: 

d^ causing the processor to implement the interface application program to determine the 
positions of the recognised words in the processing application program and to monitor changes 
in the positions of the recognised words: 

[c) ] e) causing the processor to implement the interface application program 
[process the signals] to form link data linking the audio [identifier] data to the [character 
component positions in the character string] recognised words, wherein said li nk data comprises 
the audio identifiers and the determined positions of corresponding recognise d words, and to 
update said link data in response to monitored changes in positions of the rec ognised words: 

[d) ] 0 causing the processor to generate an image of the [characters] recognised 
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words on a display; 

[e) ] g) causing the processor to receive a selection signal generated by a user for 
selecting at least one word and to identify [any] audio components corresponding to the at least 
one selected [characters] word : and 

[f) ] h) causing the processor to send the identified audio components in the order 
of the [character component] word positions in the [characters] word string to an audio play back 
device. 

_ 

Please add the new claims 63-97: 
[New] Data processing apparatus comprising: 

input means for receiving recognition data and corresponding audio data from a speech 
recognition engine, said recognition data including a string of recognised characters and audio 
identifiers identifying audio components corresponding to a character component of the 
recognised characters; 

storage means for storing said audio data received from said input means; 
processing means for receiving and processing the input recognised characters to at least 
one of replace, insert move and position the recognised characters to form a processed character 
string; 

link means for forming link data linking the audio identifiers to the character component 
positions in the character string and for updating said link data after processing to maintain the 
link between the audio identifiers and the character component positions in the processed 
character string; 

display means for displaying the characters received by said processing means; 
user operable selection means for selecting characters in the displayed characters for 
audio playback, where said link data identifies any selected audio components, if present, which 
are linked to the selected characters; 

audio playback means for playing back the selected audio components in the order of the 
character component positions in the character string or the processed character string; 
file storage means for storing the recognised characters in a file; 



m 
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means for selectively disabling one of the receipt of the recognised characters by said 
processing means and the recognition of speech by said speech recognition engine for a period of 
time, means for storing the audio data for the period of time in said storage means as an audio 
message associated with the file; and 

storage reading means for reading said file for input to said processing means, and for 
reading said audio message for playback by said audio playback means. 



•64. [New] Data processing apparatus as claimed in claim-63 wherein said storage reading 
means is controllable by a user to read said audio message at any time after said file has been 
input to said processing means until said processing means is no longer processing said file. 



-65. [New] A data processing arrangement comprising: 

a data processing apparatus, the data processing apparatus comprising: 
input means for receiving recognition data and corresponding audio data from a speech 
recognition engine, said recognition data including a string of recognised characters and audio 
identifiers identifying audio components corresponding to a character component of the 
recognised characters; 

processing means for receiving and processing the input recognised characters to at least 
one of replace, insert move and position the recognised characters to form a processed character 
string; 

link means for forming link data linking the audio identifiers to the character component 
positions in the character string and for updating said link data after processing to maintain the 
link between the audio identifiers and the character component positions in the processed 
character string; 

storage means for storing said recognition data and audio data received from said input 
means, and for storing said link data; 

display means for displaying the characters received by said processing means; 

user operable selection means for selecting characters in the displayed characters for 
audio playback, where said link data identifies any selected audio components, if present, which 
are linked to the selected characters; and 
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audio playback means for playing back the selected audio components in the order of the 
character component positions in the character string or the processed character string; 

file storage means for storing the recognised characters in a file; 

means for selectively disabling one of the receipt of the recognised characters by said 
processing means and the recognition of speech by said speech recognition engine for a period of 
time with means for storing the audio data for the period of time in said storage means as an 
audio message associated with the document; 

storage reading means for reading said document for input to said processing means, and 
for reading said audio message for playback by said audio playback means; and 

an editor work station comprising: 

data reading means for reading the characters, link data, and audio data from said data 
processing apparatus; 

editor processing means for processing the characters; 

editor link means for linking the audio data to the character component position using the 
link data; 

editor display means for displaying the characters being processed; 

editor correction means for selecting and correcting any displayed characters which have 
been incorrectly recognised; 

editor audio playback means for playing back any audio component corresponding to the 
selected characters to aid correction; 

editor speech recognition update means for storing the corrected characters and the audio 
identifier for the audio component corresponding to the corrected character in a character 
correction file; 

data transfer means for transferring the character correction file to said data processing 
apparatus for later updating of models used by said speech recognition engine; and 

audio message reading means for reading the audio message associated with characters 
being processed by said editor processing means for playback by said editor audio playback 
means; 

said data processing apparatus including correction file reading means for reading said 
character correction file to pass the data contained therein to said speech recognition engine for 
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the updating of the models used by said speech recognition engine. 

[New] A data processing arrangement as claimed in claim-65* wherein said audio 
message reading means is controllable by a user to read said audio message at any time the 
associated characters are being processed by said editor processing means. 

cm 

J>?. \TNew] A data processing method comprising the steps of: 

receiving recognition data and corresponding audio data from a speech recognition 
engine, saidVecognition data including recognised characters and audio identifiers identifying 
audio components corresponding to text components in the recognised text; 

storing thesaudio data; 

inputting the\ecognised characters to a processor for the processing of the characters to at 
least one of replace, insert move and position the characters to form a processed character string; 

forming link data limdng the audio identifiers to the character component positions in the 
characters and updating said link data after processing to maintain the link between the audio 
identifiers and the character component positions in the processed character string; 

displaying the characters input t€tthe processor; 

selecting displayed characters for aitdk) playback, whereby said link data identifies any 
selected audio components, if present, which ar^inked to the selected characters; 

playing back the selected audio components n^the order of the character component 
positions in the character string; 

storing the characters as a file; \. 

selectively disabling one of the importation of recogniseabtiaracters into the processor 
and the recognition of speech by said speech recognition engine for a period of time; 

storing the audio data for the period of time as an audio message a^ciated with the file; 

at a later time, reading said file for input to the processor; and \ 

allowing a user to select whether to read and playback said audio message'associated with 
said file. \ 
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[New] A method as claimed in claim^? wherein said audio message can be read and 
played back at any time said file is open in the processor. 

,69. [New] A method of processing data comprising: 
at an author work station: 

receiving recognition data and corresponding audio data from a speech recognition 
engine, said recognition data including recognised characters and audio identifiers identifying 
audio components corresponding to text components in the recognised text; 

storing the audio data; 

inputting the recognised characters to a processor for the processing of the characters to at 
least one of replace, insert move and position the characters to form a processed character string; 

forming link data linking the audio identifiers to the character component positions in the 
characters and updating said link data after processing to maintain the link between the audio 
identifiers and the character component positions in the processed character string; 

displaying the characters input to the processor; 

selecting displayed characters for audio playback, whereby said link data identifies any 
selected audio components, if present, which are linked to the selected characters; and 

playing back the selected audio components in the order of the character component 
positions in the character string; wherein the characters, the link data, and the audio data are 
stored; and 

at an editor work station: 

obtaining the stored characters, link data and audio data from the author work station; 
inputting the characters into a processor; 

linking the audio data to the character component positions using the link data; 
displaying the characters being processed; 

selecting any displayed characters which have been incorrectly recognised; 
playing back any audio component corresponding to the selected characters to aid 
correction; 

correcting the incorrectly recognised characters; 

storing the corrected characters and the audio identifier for the audio component 
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corresponding to the corrected character in a character correction file; and 

transferring the character correction file to the author work station for later updating of 
models used by said speech recognition engine; 

wherein, at a later time, said character correction file is read at said author work station to 
pass the data contained therein to said speech recognition engine for updating of said models; 

wherein, at said author work station, storing the characters as a file; 

selectively disabling one of the importation of recognised characters into the processor 
and the recognition of speech by said speech recognition engine for a period of time; 

storing the audio data for the period of time as an audio message associated with the file; 

and 

at a later time, reading said file for input to the processor; and 
at said editor work station, reading the audio message associated with the file being 
processed by the processor, and playing back the read audio message. 

Jti. [New] A method as claimed in claiiru^ wherein the audio message can be read and 
played back at any time said file is open in the processor. 

j)f. ^\H$w] A universal speech-recognition interface that enables operative coupling of a 
speech-recogm!tenengine to at least any one of a plurality of different computer-related 
applications, the univerSal^eech-recognition interface comprising: 

input means for receivin^speech-recognition data including recognised words; 
output means for outputting the retegnised words into at least any one of the plurality of 
different computer applications toaHw^processing of the recognised words as 
input text; and 

audio playback means for playing audio data associated with tiJ&^re^ognised words. 



7, 
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\New] The universal speech-recognition interface of claim-Tl 9 further comprising: 
mbans, independent of the one computer application, for forming link data linking a 

\ portion of the audio data to at least one the recognised words independently of the 
o&e computer application, the link data comprising: 

oneW more audio identifiers which link a portion of the audio data to one or more 

\ecognised words; and 
one or mote position identifiers which link the recognised words to corresponding 
positions within the one computer application; and 
means, independent ofthe one computer application, for updating the position identifiers 
in response to chahges in positions of the recognised words within the one 
computer application^ 

\ *° 

[New] The universal speech-recognition interface of claim*?! further comprising: 
user operable selection means for selecting one or more of the recognised words in the 

one computer application, wher^n the audio playback means is responsive to the 

selection to playback associated auisjio data. 

J7< [New] The universal speech-recognition interface of claim 2\ wherein the plurality of 
different computer applications includes a wordprocessingV>plication and at least one of a 
spreadsheet processing application, an electronic-mail application, a presentation application, and 
a computer-aided-design application. \ 

J*&. [New] A speech-recognition interface that enables operative coupling of a speech- 
recognition engine to a computer-related application, the interface comprising: 

input means for receiving speech-recognition data including recognised words; 
output means for outputting the recognised words into a computer application to allow 
processing ofthe recognised words as input text, including changing positions of 
the recognised words; and \ 
means, independent ofthe computer-related application, for determining positidra; ofthe 
recognised words in the computer-related application. \ 

B \ 
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^76. [New] The speech-recognition interface of clainrTs, further comprising: 

means, independent of the computer-related application, for monitoring changes in 
positions of the recognised words in the computer-related application. 

j 

[NeV<] The speech-recognition interface of claim-?3 5 further comprising: 
means, lhctependent of the one computer application, for forming link data linking a 

portion^ the audio data to at least one of the recognised words independently of 
the computer^dated application, the link data comprising: 
one or more audicritk{itifiers which link a portion of the audio data to one or more 
recognised words; 

one or more position identifiers whkh link the recognised words to corresponding 
positions within the computer-refete(^application; and 
means, independent of the one computer application, for&pd^ting the position identifiers 
in response to changes in positions of the recognised words^khin the one 
computer application. 

7&. [New] The speech-recognition interface of claim-75, further comprising: 

audio playback means for playing audio data associated with the recognised words. 

P*t. [NeWJJ^ie universal speech-recognition interface of claim*78, further comprising: 

user operable selecfioirmeans4^^ one or more of the recognised words in the 

computer-related application, whereintiielultfrehp^ is responsive to 

the selection to playback associated audio data. 

[New] Data processing apparatus as claimed in claim 1 wherein said interface application 
program means is operative to determine and monitor the positions of the recognised words by 
determining and monitoring the position of a first letter of each of the recognised words in text of 
said processing application program means, and said link data comprises the audio identifiers and 
the determined positions of the first letter of corresponding recognised words. 
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jtf . [New] A data processing arrangement as claimed in claim-*3 wherein said interface 
application program means is operative to determine and monitor the positions of the recognised 
words by determining and monitoring the position of a first letter of each of the recognised words 
in text of said processing application program means, and said link data comprises the audio 
identifiers and the determined positions of the first letter of corresponding recognised words. 

.4 .. . 

£± [New] Data processing apparatus as claimed in claim 1 further comprising processing 
means operative under the control of a computer operating system, wherein said interface 
application program means comprises an interface application program implemented from within 
said computer operating system, said processing application program means comprises a 
processing application program implemented from within said computer operating system, and 
said interface application program is operative to determine and monitor the positions of the 
recognised words using operating system functions communicated via the computer operating 
system. 

j&. [New] Data processing apparatus as claimed in claim-R further comprising processing 
means operative under the control of a computer operating system, wherein said interface 
application program means comprises an interface application program implemented from within 
said computer operating system, said processing application program means comprises a 
processing application program implemented from within said computer operating system, and 
said interface application program is operative to determine and monitor the positions of the 
recognised words using operating system functions communicated via the computer operating 
system. 

M. [New] A data processing method as claimed in claim-23 wherein the positions of the 
recognised words are determined and monitored by determining and monitoring the position of a 
first letter of each of the recognised words in text of the processing application program, and said 
link data comprises the audio identifiers and the determined positions of the first letter of 
corresponding recognised words. 

7^ 
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J8i. [New] A method as claimed in claim wherein the positions of the recognised words 
are determined and monitored by determining and monitoring the position of a first letter of each 
of the recognised words in text of the processing application program, and said link data 
comprises the audio identifiers and the determined positions of the first letter of corresponding 
recognised words. 

JS6. [New] A method as claimed in claim -23 wherein the interface application program and 
the processing application program are both implemented from within a computer operating 
system, and the positions of the recognised words in said processing application program are 
determined and monitored using operating system functions communicated via the computer 
operating system. 

gs ft 

[New] A method as claimed in claim -35 wherein the interface application program and 
the processing application program are both implemented from within a computer operating 
system, and the positions of the recognised words in said processing application program are 
determined and monitored using operating system functions communicated via the computer 
operating system. 

[New] Data processing apparatus comprising 

input means for receiving recognition data from a speech recognition engine and 
corresponding audio data, said recognition data including a string of recognised words and audio 
identifiers identifying audio components corresponding to each of the recognised words; 

processing means for implementing an interface application program which receives the 
input recognised words, inputs the recognised words into a processing application program to 
process the input recognised words to cause the recognised words to be moved, and forms link 
data linking the audio data to the recognised words, said link data comprising the audio 
identifiers and information identifying the corresponding recognised words; 

display means for displaying the words received and processed by said processing 
application program; 

user operable selection means for selectively identifying a word in the displayed words, 
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wherein said interface application program is operative to compare the identity of the selected 
word with said link data to identify any corresponding audio component; and 

audio playback means for playing back any identified corresponding audio component. 

s89. [New] Data processing apparatus as claimed in claim-8* including storage means for 
storing said link data, and said audio data. 

£0. [New] A data processing method comprising: 

inputting recognition data from a speech recognition engine and corresponding audio 
data, said recognition data including a string of recognised words and audio identifiers 
identifying audio components corresponding to each of the recognised words; 

inputting the recognised words to a processor implementing an interface application 
program to receive the input recognised words, to pass the recognised words to a processing 
application program for processing the recognised words to cause the recognised words to be 
moved, and to form link data linking the audio data to the recognised words, said link data 
comprising the audio identifiers and information identifying the corresponding recognised words; 

displaying the recognised words input to and processed by the processor application 
program; 

selectively identifying a word in the displayed words; 

using the interface application program to compare the identity of the selected word with 
said link data to identify any corresponding audio component; and 
playing back any identified corresponding audio component. 

J$\ . [New] A method as claimed in claim 90 including storing the audio data and the link 
data. 
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s&t. [New] A computer usable medium having computer readable instructions stored therein 
for causing a processor in a data processing apparatus to process recognition signals defining a 
string of recognised words and corresponding audio data to display the words and selectively 
play the audio data, the instructions comprising instructions for: 

a) causing the processor to input the recognition signals from a speech recognition 
engine and the audio data, the recognition signals including a string of recognised words and 
audio identifiers identifying audio components corresponding to each recognised word; 

b) causing the processor to implement an interface application program to receive the 
input recognised words and to input the recognised words into a processing application program 
to process the recognised words to cause the recognised words to be relatively moved; 

c) causing the processor to implement the interface application program to form link 
data linking the audio data to the recognised words, said link data comprising the audio 
identifiers and information identifying the corresponding recognised words; 

d) causing the processor to generate an image of the recognised words on a display; 

e) causing the processor to receive a selection signal generated by a user for 
selectively identifying a word in the displayed words; 

f) causing the processor to implement the interface application program to compare 
the identity of the selected word with said link data to identify any corresponding audio 
component; and 

g) causing the processor to send the identified corresponding audio component to an 
audio playback device. 



S3. [New] A computer usable medium as claimed in claim-92 wherein the instructions 
include instructions for causing the processor to store said link data and said audio data. 
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J)A. [New] Data processing apparatus comprising: 

input means for receiving recognition data from a speech recognition engine and 
corresponding audio data, said recognition data including a string of recognised words and audio 
identifiers including audio components corresponding to each recognised word; 

storage means for storing the audio data received from said input means; 

processing means operative under the control of an operating system to implement a first 
application program which receives the input recognised words in text positions, and which 
processes the recognised words such that the positions of the recognised words are changed to 
form a processed word string, and a second application program which determines the positions 
of and monitors changes in the positions of the recognised words in said first application 
program using operating system functions communicated via the computer operating system, and 
which forms link data linking the audio data to the recognised words and updates said link data 
in response to monitored changes in the positions of the recognised words, said link data 
comprising the audio identifiers and the determined positions of corresponding recognised 
words; 

display means for displaying the recognised words; 

user operable selection means for selecting at least one word in the displayed words, 
wherein said second application program is operative to identify any selected audio components, 
if present, which are linked to the at least one selected word; and 

audio playback means for playing back any selected audio component. 

lie 7^* 

J)5. [New] Data processing apparatus as claimed in clainv-94 including means operable by a 
user to allow the selection of said second application program from amongst a plurality of 
application programs implementable within the computer operating system. 
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£6. [New] A data processing method comprising: 

inputting recognition data from a speech recognition engine and corresponding audio 
data, said recognition data including a string of recognised words and audio identifiers 
identifying audio components corresponding to each of the recognised words; 
storing the audio; 

implementing a first application program within a computer operating system to receive 
the input recognised words in text positions, and to process the recognised words such that the 
positions of the recognised words are changed to form a processed word string; 

implementing a second application program from within the computer operating system 
to determine the positions of the recognised words and monitor changes in the positions of the 
recognised words in the first application program using operating system functions 
communicated via the computer operating system, to form link data linking the audio data to the 
recognised words, and to update the link data in response to monitored changes in the positions 
of the recognised words, wherein said link data comprises the audio identifiers and the 
determined positions of corresponding recognised words; 

displaying the recognised words; 

selecting at least one word in the displayed words, wherein the second application 
program identifies any selected audio components, if present, which are linked to the at least one 
selected word; and 

playing back any selected audio component. 

£T. [New] A method as claimed in claim -96 including selecting the second application 
program from amongst a plurality of possible application programs implementable within the 
computer operating system. 
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